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On  the  Use  of  a  Cumulative  Distribution  as  a  Utility 


Function  in  Educational  or  Employment  Selection 


James  J.  Chen  and  Melvin  R.  Novick 
The  University  of  Iowa 


Abstract 

Formal  decision  theory  can  make  important  contributions  to  educational 
or  employment  decision-making,  provided  one  can  quantify  the  utilities  of 
different  possible  outcomes  such  as  test  scores,  grade-point  averages  or 
other  common  outcome  variables.  Utility  is  usually  a  monotonic  increasing 
function  of  true  ability  or  performance  score.  A  cumulative  probability 
function  is  then  very  convenient  for  describing  one's  utilities.  Moreover, 
calculations  of  expected  utility  of  a  decision  is  greatly  simplified  when 
the  utility  and  the  probability  function  have  the  same  functional  form, 
e.g.  both  normal.  A  least-squares  procedure  for  fitting  a  utility 
function  is  described  and  applied  to  truncated  normal  and  beta  distribution 
functions. 


Introduction 


Bayesian  approaches  to  the  problems  of  selection  or  certification 
have  been  discussed  by  several  authors.  Cross  and  Su  (1975),  Petersen 
(1976),  and  Huynh  (1976,  1977)  use  threshold  utility  or  a  constant  loss 

function  to  derive  the  cut-off  scores  in  various  models.  Van  der  Linder, 
and  Mellenbergh  (1977)  extend  the  utility  (loss)  function  to  a  linear 
form.  Novick  and  Lindley  (1978)  suggest  using  a  parametric  utility 
function.  They  discuss  several  advantages  of  using  a  cumulative 
distribution  as  a  utility  function  and  recommend  using  the  cumulative 
normal  utility  function.  However,  one  restrictive  property  of  the 
cumulative  normal  utility  is  its  symmetry  about  the  mean.  This  can  be 
avoided  by  using  only  a  portion  of  a  symmetric  function,  e.g.  a 

truncated  normal  function,  or  using  a  non-svmmetric  function,  o.g.  a  beta 
function. 

In  this  paper  we  consider  using  a  truncated  normal  or  a  generalized 
beta  cumulative  distribution  function  (cdf)  as  a  utility  function.  We  will 
refer  to  them  as  a  truncated  normal  utility  (TNU)  and  a  generalized  beta 
utility  (GBU)  respectively.  Applications  of  the  two  utility  functions  will 
be  discussed  specifically.  A  TNU  and  a  GBU  both  have  the  advantage  of  being  cumul 
tlve  distribution  functions  and  at  the  same  time  they  provide  increased  f^.vibilit 
in  applications.  For  example,  it  can  be  symmetrical  between  risk-aversion  and 
risk-proneness  or  it  can  be  risk-averse  or  risk-prone  throughout  its  range. 
Moreover,  if  the  utility  and  the  posterior  distribution  have  the  same  functional 
form,  both  normal  or  both  beta,  then  the  computations  of  the  expected 
utility  become  simple. 

The  major  problem  in  utilizing  utility  theory  is  the  assessment  of  the 
utility  function.  Procedures  for  obtaining  a  utility  function  have  been 
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given  by  Mostellor  and  Nogeo  (1951),  I’ratt,  Haiffa  and  Schlaifer  (1965), 
Schlalfer  (1969),  and  Keeney  and  Ralffa  (1976).  One  of  the  difficulties 
In  utility  assessment  is  resolving  the  incoherence  of  subjects.  In 
resolving  subject  incoherence,  one  can  require  a  subject  to  give  more  than 
the  minimum  number  of  judgments  in  fitting  a  utility  function.  In  this 
paper  we  discuss  a  procedure  for  fitting  a  cumulative  utility  function  based  on 
fixed  state  least-squares  utility  assessment  as  given  by  Novick  and 
Lindley  (1979). 


L 
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A  Truncated  Normal  Utility 


Consider  the  problem  of  educational  or  personnel  selection. 

Let  0  denote  the  ability  (or  a  measure  of  performance)  of  an  applicant 

and  U(8)  be  the  utility  for  selecting  an  applicant  with  ability  6.  Assume 

the  range  of  8  is  (0  ,  0  ],  then  a  truncated  normal  utility  (TNU)  function 

on 

is  defined  by 


0  <  0  <  0  (1) 
o  —  —  n 


where  p  and  a  are  respectively  the  mean  and  standard  deviation  and 
♦  (•)  is  the  standardized  normal  distribution  function.  Hence,  a  TNU 
uses  only  a  portion  of  a  normal  cumulative  function.  The  utilities  at 
Che  two  end  points  are  0  and  1. 

To  investigate  the  behaviors  of  a  person  from  the  utility  function, 
Pratt  (1964)  develops  the  risk  aversion  function  r(0)  for  a  measure  of  risk 
aversion  of  a  utility  function,  where 


.(9)  .  -  £Vin  .  __dUj£l 

'  A  Pit  '  A  A 


If  r(0)  is  positive  at  0  ■  0  *  then  U(0)  is  risk-averse  at  0  ; 
if  r(0)  is  negative  at  0'  then  U(6)  is  risk-prone  at  6*;  and  if  r(0) 
is  zero  at  0'  then  U(0)  is  risk  neutral  at  0*.  The  risk  aversion 
function  for  the  TNU  defined  in  (1)  is 


r(0)  - 


9  <  e  <  c 

o  —  --  n 
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Thus,  the  sign  of  r(0)  depends  on  the  position  of  p.  If  6q<  p  ,  then 
r(0)  is  positive  for  all  0  ,  U(0)  is  risk-averse  through  its  range. 
If  0n>p, then  r(0)  is  negative  for  all  0  ,  U(0)  is  risk-prone  through 
its  range.  If  0q<  p  <  Qn  tVien  r(0)  is  negative  for  0  <  p  and  positive 
for  0  >  u,  U(0)  is  risk-prone  for  low  e  values  and  risk-averse  for 

high  0  values.  This  property  is  true  for  most  unlmodal  distribution 
functions.  Since  the  derivative  of  the  risk  aversion  function  in  the 

normal  case  is 


r'(0) 


1 

"52  • 


This  implies  that  U(0)  has  strictly  increasing  risk  aversion.  This 

characteristic  is  particularly  useful  in  educational  selection. 

For  0  <p  <0  ,  a  TNU  reflects  a  decision  maker  who  is  risk-taking  for 

o  n 

low  ability  0, and  risk-avoid  for  high  ability  0;  and  his  willingness 
to  take  the  risk  decreases  as  0  increases.  This  seems  reasonable  and 
common  in  educational  selection. 

Finally,  for  ®0  <  y  <  the  limiting  forms  of  the  TNU  become 

very  simple.  If  then 

* 

0(0)  =  1  for  6  >  p 

0  for  0  <  p  . 


This  is  a  threshold  utility  with  threshold  point  at  0=p.  And  if  ~  -  00 
then 


U(9)  -  — ^ 

V  °o 


0  <  0  <  0 
o  —  —  n 


.-1 


The  utility  becomes  a  linear  function  with  slope  (0  -  0Q) 

(This  result  also  holds  for  P  >  O  or  P  <  ©0  ) .  Thus,  for  0  "  ' 

the  function  results  in  a  smooth  curve  so  that  the  lower  portion 
(9  <  u  )  is  convex  and  the  upper  portion  (0  >  p  )  is  concave  with 


the  point  of  inflection  at  0  *  p. 


'Vi  l  .  Ot)‘>L'  ■  .'l-V.  .  'I 


U(0)  =  j 


u(0)  (1  F(o  |  X) 


where  F(6  |  x)  is  the  posterior  distribution  of  0  given  observed  score  x.  The 
Bayes  rule  is  to  select  a  candidate  with  the  highest  expected  utility. 
Therefore,  the  computations  of  the  expected  utility  become  important 
in  the  selection  analysis. 

As  Lindley  (1977)  and  Novick  and  l.indley  (1978)  indicate, 
if  one  can  match  the  function  form  of  utility  U(0)  with  the  (posterior)  dis¬ 
tribution  F(6  |  x)  then  the  computations  of  expected  utility  become  simple. 

In  most  applications  F(0  |  x)  or  can  be  approximated  by  a  normal  distribution 

Huynh  (1979).  Assume  F(0  |  x)  is  a  normal  distribution  with  mean  g  and 

o 

2 

variance  aQ,  on  [0o>  0^]  then  the  expected  utility  of  the  TNU  in  (1)  is 


E  U(0) 


I - 1 2 _J -  d  4»( - -} 

0  (b  -  a)  (b  -  a  )  '  °o  ' 

o  o  o 


(b  -  a)(b  -  a  ) 

o  o 


b  -  a 


0  <  0  <  0 

o  —  —  n 


,  0  <  0  <  0 

o  —  —  n 


Pr  (  £  -  0  10) 

(b-a) (b  -  a  ) 
o  o 


a  ,  0  <  G  <  0 

irrr  0  ~  ~  n 


Pr(£  -  0  <  o,  e  <e<e) 
—  o  —  —  n 

(b  -  a)  (b  -  a  ) 
o  o 


b-a 


where  a  *  ♦ 


’0O  ‘  v 


) .  b  -  *(9-^) , 


z  6  u 
.  f  o  -  c 

a  ■  <t>  | — - - 

o  \  oo 


6 


-(Nr1) 


and  £  and  0  are  two  independent  random  variables  such  that 


£«vN(p,o)  and  0'\.N(po,oq)  .  Since  £  and  9  are  both  normal,  (£-0,0)  has 

a  bivariate  normal  distribution  with  mean  (  p  -  p  ,  p  )  and  covariance 

o  o 


matrix 


o2  + 


0 

o 


-  cr 


-  o 


o 

.2 


The  expected  utility  can  be  obtained  by  computing  the  probability 

of  the  bivariate  normal  variable  over  the  region  £  -  0  <  0  arri  0  <  0  <  0 

—  o—  —  n 

Tables  for  bivariate  normal  probabilities  are  compiled  by  the  (U.S.) 

National  Bureau  of  Standards  (1959).  If  b  -  a^l,  e.g. 

o  o 

0o  £  PQ~  4oo  <  pq  +  ,  then  the  expected  utility  in  (3)  is 


E  u(9)  »  ?r(L  ^  -  r-S~ 

b  -  a  b  -  a 

c  -  a 
b  -  a 

where 

•  -  * 

and  a,  b,  c  may  be  obtained  from  a  univariate  normal  distribution 
table.  A  computer  algorithm  for  computing  bivariate 
normal  probabilities  is  given  by  Divgi  (1979). 
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In  the  previous  discussions  0  is  considered  to  be  on  the  interval 

[8,6],  in  some  cases  0  may  be  defined  beyond  [9  .  9  ]•  For  example, 
o  n  on 

in  latent  trait  models  the  range  of  9  is  (-”,“).  In  this  case,  we  can 

assign  U(0)  =  1  for  0 >  0  and  U(o)  =  0  for  0  <  0  .  Then  the  expected 

a  o  r 

utility  in  (3)  with  F(0  |  x)  defined  over  (-«  ,  «)  becomes 

Pr(£  -  0  <.0,  0  <_  0  <_  0  )  -  a 

E  U<6>  -  - (b"~  "~a)'  ° - +  (1  -  V-  (3) 

where  £,  0,  a  ,  b  and  bQ  are  as  previously  defined. 

In  the  problem  of  setting  cut-off  (passing)  scores,  it  is  necessary 

to  require  monotonic  expected  utility.  More  precisely,  for  a  given 

increasing  utility  function  U(9)  the  expected  utility  should  be  a 

monotonic  function  of*,  the  observed  score.  A  sufficient  condition  for 

monotonic  expected  utility  is  that  the  posterior  distribution  is 

stochastically  increasing  in  x,  i.e.  if  x < x'  implies  F(0  |x) >  F(0  |  x') 

for  all  0  (Chuang,  Chen  and  Novick,  1981).  It  is  easy  to  show  that  if  the 

mean  V  of  the  normal  distribution  F(0  j  x)  is  an  increasing  function  of  x 
o 

and  the  variance  <Jq  is  independent  of  x  then  F(0  |  x)  is  stochastically 
increasing  in  x.  So  the  expected  utility  in  (3)  or  (3)'  will  increase 


as  x  increases. 


8 


We  now  give  an  example  of  applying  the  TNU  to  the  regression 

2 

model  (Petersen  and  Novick  1976)  y=a  +  8(x-x)+e  where  e-'-NfO,  o  ), 
in  the  selection  of  applicants.  The  example  is  based  on  the  data  used 
in  Petersen's  (1976)  model  for  selection  under  restrictions.  The 
criterion  variable  Y  is  the  first  semester  college  grade-point  average. 
The  ACT  composite  score  is  used  for  the  predictor  variable  X.  The 
applicants  can  be  divided  into  disadvantaged  and  advantaged  groups. 

The  range  of  Y  is  from  0  to  4  and  the  range  of  X  is  from  0  to  36. 

Table  1  gives  the  sample  size  of  N,  mean  x  and  y,  standard  deviation 
sx  and  Sy,  correlation  r^  and  regression  coefficient  3  for  each  group. 

Table  ]  near  here 


Assume  indifference  prior  distributions  of  «,  3  andufor  each  group, 

then  the  posterior  predictive  distribution  for  y  given  an  applicant  with 

score  x  is 
o 


y  -  [y  +  8  (xq  -  x)  ] 

[N+l  .  (xo  -  xT2^  ~  *N-2 
S  LlT+  E(x~-  x)2j 


W 


2  2  2 

where  s  **  N  s  (1  ~  rw)/(N  -  2)  is  the  mean  square  error  of  the 

y  *y 

regression  equation.  If  N  is  large  then  the  t  distribution  in  (4) 
is  approximated  by  a  normal  variable  and 


N+l 

N 


+ 


x) 

(x-x) 


■>  1  . 
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Therefore,  the  posterior  predictive  distribution  for  y  for  a  person 
with  score  xq  approximates  to  a  normal  distribution  with  mean 
y  +  B  (xQ  -  x)  and  variance  s  .  For  a  positive  value  of  B,  the 
mean  is  an  increasing  function  of  xq,  and  the  variance  is  independent 
of  xq,  so  the  expected  utility  is  increasing  in  xq. 

Suppose  the  utility  for  disadvantaged  group  has  mean  1.5  and 
variance  2  and  the  utility  for  advantaged  group  has  mean  2  and 
variance  1.5,  the  expected  utilities  computed  from  (3)  of  each  group  are 

given  in  Table  2.  It  is  easy  to  see  that  of  two  applicants  from 
different  groups  with  the  same  test  score,  the  applicant  from  the 
disadvantaged  group  is  preferred.  For  the  given  data  set  and  utility 
functions.  Table  2  can  be  used  for  the  selection  of  applicants. 


Table  2  near  here 
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A  Generalized  Beta  Utility 


We  now  consider  the  use  of  a  generalized  beta  cumulative  distribution 
as  a  utility  function.  Again,  assume  the  range  of  6  is  A 

generalized  beta  utility  is  defined  as 


r  r(a  +  b) 

U(6)  -  \  - 

^  r(a)r(b) 


(x  -  e  )a_1(e  -  x) 
o  n 

<e„  -  eo)*tb“1 


b-1 


d  x 


e  <  e  <  e  (5) 

o  —  —  n 


The  standard  beta  distribution  has  0  *  0  and  0  =  1.  The  risk  aversion 

o  n 

function  for  a  GBU  is 


r(0) 


b-1 

TvTT 


a  -  1 

«>  -  e„> 


0  <  0  <  0 

o  —  —  n 


Taking  the  derivative  to  find  where  the  GBU  has  constant,  decreasing  or 
increasing  risk- aversion,  we  have 


r*(0) 


b-1  +  a  -  1 

(Q  -  0) 2  (0  -  o y 

n  o 


0  <  0  <  0 

o  —  —  n 


With  the  choices  of  a  and  b,  beta  utility  provides  a  wide  variety 

of  utility  functions.  For  example,  for  a<  1  and  b<  1  the  GBU  is  risk 

averse  when  6  is  less  than  0a>  and  is  risk  prone  when  0  is  greater  than 

0  ,  where  0  =  [(a  -  1)  0  -  (b  -  1)  0  J/  (a  +  b  -  2)  is  the  antimode  of 

33  n  o 

the  generalized  beta  function.  It  has  strictly  decreasing  risk  aversion. 

For  a  < 1  and  b-1  the  beta  utility  is  risk  averse  through  its  range  and 
it  has  strictly  decreasing  risk  aversion.  For  a<  1  and  b>  1  the  beta 
utility  is  risk  averse  through  its  range  and  it  has  decreasing  risk  aversion 
when  0<0^  and  it  has  Increasing  risk-aversion  when  0  >  0fe  where 


9b  "  (0n  *  eoy''(a'>l))  '  l1  +  J’  -(TTT)-)  '  A  ^creasing  risk-averse 

utility  function  is  useful  in  economic  applications. 
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In  the  previous  section,  we  discussed  some  properties  of  the  TNU 

function.  A  GBU  with  a_>  1  and  b>_  1  has  similar  characteristics.  It  is 

increasing  risk  aversion.  For  a  >1  and  b  >1  the  beta  utility  is  risk  prone 

when  6  >  0  and  is  risk  averse  when  6  <  0  ,  6  is  the  mode  of  the 
a  a  a 

generalized  beta  function.  For  a  >1  and  b  ■  1  the  beta  utility  is  risk 
prone  through  its  range.  For  a  -  1  and  b  >1  the  beta  utility  is  risk 
averse  through  its  range.  The  last  two  types  represent  the  lower  half 
curve  and  the  upper  half  curve  of  a  normal  utility  respectively.  Finally, 
for  a  «  1  and  b  -  1  the  beta  utility  is  equivalent  to  a  TNU  with  o  *  «. 

We  summarize  the  above  results  in  Table  3. 


Table  3  near  here 

The  expected  utility  of  a  GBU  can  be  computed  by  simple  numerical 
integration.  If  a  and  b  are  integers,  the  beta  utility  in  (5)  becomes 
a  polynomial  utility  with  degree  a  +  b  -  1.  Then  expected  utility  is  the 
linear  combinations  of  the  moments  of  the  posterior  distribution  F(o|x). 
The  moments  for  many  common  probability  distributions  are  tabulated  and 
could  be  used  in  the  calculations  of  the  expectation  of  a  polynomial 
utility. 

If  F(0  |  x)  has  a  beta  distribution  then  the  expected  utility  becomes 

one  beta  variable  less  than  or  equal  to  another  beta  variable.  Assume  £ 
and  0  are  two  independent  beta  random  variables  with  distributions  U(0) 

and  F(0  f  x)  respectively.  Then  the  expected  utility  of  a  GBU  in  (5)  is 


E  U(0)  -  Pr  (£  <  0) 


0  -  0o 

Note  that  the  beta  variable  0  is  also  defined  on  (0  ,  0  J.  Let  £'  ■  — - -  and 

on  8-8 

6  -  0o  no 

6'  a  -g  —  -g  ,  then  £'  and  0'  are  usual  beta  random  variables  defined  on  [0,1]  and 
n  o 


Hence 


E  U  (6)  -  Pr  (C'<  O'). 


(6) 


The  probability  on  the  right  hand  aide  does  not  depend  on  0Q  or  0n- 
A  formula  for  computing  this  probability  is  given  in  Althara  (1969)  and 
Wei8berg  (1972). 

We  now  provide  an  example  of  the  use  of  the  beta  utility  when  the 
test  scores  follow  a  beta-binomial  model.  This  model  has  been  used 
extensively  in  the  theory  of  criterion-referenced  testing  by  Huynh  and 
others.  For  a  test  of  n  items,  given  an  applicant  with  ability  0  the 
probability  that  he  answers  x  items  correctly  is  a  binomial  distribution. 
Assume  a  person's  ability  has  a  beta  distribution  with  parameters  p  and  q. 
Then  the  posterior  distribution  of  0  given  x  is  a  beta  (p  +  x,  q  +  n  -  x) . 
If  a,  b,  p,  q  are  integers,  Altham  (1969)  provided  the  following  formula 


tor  computing 


■  - z 


E  U(0)-  y 

s  ■  max  {p  +  x  -  b,0] 


P  +  x  -1  (a  +  p  +  x-l\  fb+q+n-x-l\ 

r -  \ _ s  /  Vp  +  q+  n-  s-l/ 

2. _  /a  +  b  +  p  +  q  +  n-  2\ 

{p  +  x  -  b,0]  \  p  +  q  +  n-1  / 


Again,  consider  the  selection  on  the  restricted  model.  Suppose  the 
utility  for  the  disadvantaged  group  has  beta  cumulative  function  with 
parameters  2  and  4  and  the  utility  for  the  advantaged  group  has  beta  cumulative 
function  with  parameters  2  and  2.  Assume  indifference  prior  (Novick  and 
Jackson  1974,  p.  156)  for  each  group,  i.e.  p  -  q  -  0.  For  a  teat  of  16  items 
the  expected  utility  of  each  group  is  given  in  Table  4.  It  is  easy  to  see 
that  the  expected  utility  is  an  increasing  function  of  test  score.  In  fact, 
the  expected  utility  preserves  the  shape  of  the  utility  function.  For  example, 
for  the  advantaged  group  the  expected  utility  is  symmetric  at  the  score  of  8. 


Table  4  near  here 


Fitting  a  Utility  Function 


We  have  seen  above  that  there  are  some  advantages  in  using  the 
cumulative  normal  or  beta  distribution  function  as  a  utility  function. 

In  this  section  we  consider  a  least-squares  procedure  given  in  Novick  and 
Lindley  (1979)  to  fit  the  parameters  (t^Tj)  a  utm*y,  where  ^T1*T2^ 
are  (u,o)  for  a  TNU  and  (a.h.)  for  a  GBU. 


Consider  (n+1)  points  with  0  <0,  <  ...<0  and  let  U(0  )  -  U 

O  1  I\  '  i 

i  -  0,1,  ....  n.  For  a  given  triplet  (e^  0^  ,  ,  a  subject  is 

asked  to  compare  0  for  sure  with  a  gamble  on  0.  and  0.  where  i<j< 

J  X  |C 

Specifically,  the  subject  is  required  to  state  a  probability,  p 

ijk’ 

such  that  he  is  indifferent  between  0^  for  sure  and  the 
gamble:  chance  p±Jk  of  0k  and  1  -  PiJk  of  0^  Then 


UJ  ■  "ijk  \  +  (l  ljk>  U, 


or 


P 

l^P 


i^k 

ijk 


(i)  If  U(0)  is  a  TNU  function,  then 
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i 

*4 


4 


(il)  If  U(8)  is  a  GBU  function,  then 

/()  -  0  \ 
‘  n  r\ 


*i1k 
1  -  p 


[-r-rf) 

»  n  n 


ijk 


.■04W  -  (^) 


where  U'(*)  is  a  standard  beta  distribution  function,  i.e. 

"'O'*  ’  rT£f7$  f  x‘“1  (1-x>b'1  'ix-  0  1 6  -  1 


Two  values  of  are  sufficient  to  determine  and  x2*  For  checking 

the  coherence  subject  is  required  to  assess  more  than  two  indifference 
probabilities  p^^*  Making  the  same  assumption  as  in  Novick  and 
Lindley  (1979)  about  the  use  of  log-odds,  a  utility  can  be  fitted  by 
minimizing  the  following  sum  of  squares  with  respect  to  and  t2 


E  (log  T- 
ijk  v 


UJ  -  ul-) 
U,.  -  / 


-  log 


ijk 


1 
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An  example  of  using  the  least-squares  procedure  to  fit  a  TNU  and  a  GBU 
for  a  state  university  administrator  on  grade  point  average  (CPA) 
is  given  below.  The  range  of  GPA  is  from  0  to  4,  14  different  gambles 
are  formed  from  the  9pofnts  0,0.5,  1,  1.5,  2,  2.5,  3,  3.5,  4.  The 
list  of  the  gambles  is  shown  in  column  1  to  3  of  Table  5.  (There 
are  84  possible  gambles).  For  each  gamble  the  assessed  indifference 
probabilities  are  given  in  column  4.  The  Gau3s-Ncwton  method  was  used 
to  minimize  sum  of  squares.  The  algorithm  is  given  in  the  appendix. 

The  parameters  of  the  fitted  normal  and  beta  utility  are  (1.44,  0.84) 
and  (1.72,  2.99),  respectively.  The  fitted  utilities  for  the  nine 
points  are  given  in  column  5  and  6.  Two  utilities  are  reasonably 
close  to  each  other.  The  fitted  beta  has  a  mean  of  1.46  and  a  mode  of 
1.06.  The  utilities  rise  very  rapidly  for  GPA  below  2  and  very  slowly 
for  GPA  above  3.  Perhaps,  this  is  the  state  university  regulation 
that  requires  a  continuing  average  of  1.0  to  remain  in  college  on 
probation,  an  average  of  2.0  to  maintain  regular  status  and  an  average 
of  3.0  to  take  honor  courses.  Finally,  for  the  comparison  of  the 
goodness  of  fits,  the  utilities  from  nine  points  least-squares  fit 
are  given  in  column  7.  It  also  shows  that  the  utilities  at  GPA  «  0.5, 

1,  1.5,  2  increase  most  rapidly,  and  the  three  fits  are  very  close 
for  GPA  above  1.5. 


Table  5  near  here 


Note  that  equation  (7)  is  independent  of  a  -  <M— - 1  and 

.t6n~  U\  \  a  ' 

b  ■  — - - J,  This  procedure  will  fit  the  best  normal  curve  over 

the  interval  [0,0].  However,  if  p  is  far  from  the  midpoint  of  0 
on  o 

and  0^  or/and  a  is  large  then  the  performance  of  the  fitted  procedure 
will  decrease.  We  briefly  examine  this  difficulty  by  using  the  Monte 
Carlo  methods  for  the  model 


log 


lilL 


i-p 


ijk 


log 


•<V1 

M 

fei  - 
i  0  ) 

H 

+  V 


14  gambles  are  used  for  each  fit,  the  (i,  j,  k)  are  generated  uniformly 
from  the  84  gambles  and  the  error  v  from  independent  N(0,  4)  popula¬ 
tion.  Table  4  gives  the  average  of  25  fitted  mean  and  standard  deviations 
for  each  combination  of  p=2,  2.5,  3,  4  and  o=0.5,  1,  1.5,  2.  The  standard 
error  of  the  averages  are  also  provided.  They  are  indicated  in 
parenthesis.  We  consider  the  biases  ( | fitted-true| )  and  standard 
errors  of  the  fitted  values  as  a  measure  of  the  performance  of  the 
procedure. Table  4  indicates  that  an  increase  of  0  or  p  worsens  the  fit 
as  one  would  expect.  For  p  -  4  and  a*  0.5,  the  utilities  for  0O 
are  so  close  together  (close  to  0)  that  the  procedure  does  not 
converge.  This  also  suggests  that  0^  should  be  chosen  so  that  the 
utilities  of  U(0^)  increase  significantly  at  each  0^. 
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Table  6  near  here 


Fitting  a  Utility  in  Two  Groups 

The  problem  of  using  probability  distribution  function  as  a 
utility  function  arises  when  the  utilities  are  specified  for  more 
than  one  group.  In  the  previous  section,  both  utility  bounds  for 
advantaged  and  disadvantaged  groups  are  assumed  to  be  1  and  0.  This 
may  not  be  true  in  some  situations.  For  example,  of  two  applicants 
with  the  same  GPA  of  4,  one  may  prefer  the  disadvantaged  group 
applicant.  Therefore,  it  is  important  to  check  coherence  between 
groups  on  multiple  groups  assessment.  We  now  give  a  procedure  to 
exploit  coherence  between  groups.  We  will  use  GPA  as  an  example  to 
describe  the  procedure. 

Let  Up(0)  and  UA<0)  be  the  utility  function  for  disadvantaged 
and  advantaged  groups  respectively.  Suppose  selecting  a  disadvantaged 
group  applicant  with  GPA  of  4  has  the  highest  utility  of  1  and  select¬ 
ing  an  advantaged  group  applicant  with  GPA  of  0  has  the  lowest  utility 
of  0.  Hence,  UD(4)-1  and  UA(0)*0.  Consider  the  utility  for  disad¬ 
vantaged  groups ,  for  a  given  GPA  of  0^  ,  the  subject  is  asked  to 
compare  the  following  two  options: 

For  sure  disadvantaged  group  applicant  with  GPA~8^ 

and 

P  disadvantaged  group  applicant  with  GPA»4 

1- P  advantaged  group  applicant  with  GPA-0 

Let  be  the  probability  for  which  the  subject  is  indifferent  between 


the  two  options.  Then 


IS 


V»i>  -  (i  -  p4)  uACO)  +  Pl  OdC4> 

-  Pi  (8) 

for  any  i.  Assume  Up(0)  and  ( 0 )  are  two  cumulative  utility  functions 

elicited  independently  from  the  procedure  described  in  the  last  section. 
Because  Up(0)  and  U^(0)  are  two  utility  functions  for  the  same  population, 
we  have 


%<«>  ■  “d  +  sd  Ve> 

-  <1  -  sD)  +  eD  uD<e>  (9) 

The  last  equation  follows  from  0^(4)  ■  Ujj (4)  *  1.  Substituting  (8) 
into  (9)  and  simplifying  gives 

a  -  Pl)  -  sD  (i  -  uD(ei) )  <L0) 

BD  can  be  solved  immediately  from  and  Ujj(0^).  Note  that  for 
coherence,  should  be  less  than  or  equal  to  1  (or  U(G^)  should  be 
less  than  or  equal  to  p^).  Equation  (10)  is  a  linear  regression 
equation  without  intercept.  By  assessing  at  different  points, 
the  least  squares  procedure  for  estimating  cai  be  used  for  checking 
coherence.  Similarily,  the  utility  for  advantaged  group  is 

SA(0>  -  ba  ua(0)  o  <  eA  <  i 

And  $A  can  be  estimated  from  the  regression  line 

'r«*W-  <u> 

where  p^  is  the  indifference  probability  between  the  two  options: 
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For  sure  advantaged  group  applicant  with  GPA*6^ 

and 

p  disadvantaged  group  applicant  with  GPA«4 

1-p  advantaged  group  applicant  with  GPA**0  . 

If  ($D  and  SA  are  the  least  squares  solutions  from  equation  (10)  and  (11) 
then  the  assessed  utility  fox  the  two  groups  are 

ye)  -  (1  -  Bd)  +  6d  ud(6) 

and 

»A(e>  ■  *A  VM  • 

Since  the  two  bounds  of  Up (6)  and  UA(e)  are  not  0  and  1,  they  cannot  be 
considered  as  a  cumulative  distribution  function;  however,  they 
retain  all  the  properties  of  a  TNU  described  above.  The  expected 
utilities  can  be  computed  directly  from  E(Up(0))  and  E(UA(0)) 
respectively,  i.c. 

E  UD(0)  -  (1  -  SD)  +  Bd  E  UD(0) 

and 

E  UA(0)  -  BA  E  UA  (0). 
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Note  we  assume  that  the  highest  utility  and  the  lowest  utility 
are  in  two  different  groups,  i.e.  UD(4)  =  1  and  UA(0)  3  1-  Suppose 
the  two  extreme  utilities  0  and  1  occur  in  the  same  group.  Without 
loss  of  generality,  it  could  alternatively  he  assumed  that  UA(4)  -  l 
and  5^(0)  =  0,  then  only  the  utility  for  the  disadvantaged  group  has 
to  be  rescaled.  Following  the  same  arguments  described  above  we  have 


“D  +  SDUD 


0<aD  ,0<6D  <  1 


(12) 


where  p^  is  the  indifference  probability  for  the  two  options 

For  sure  disadvantaged  group  applicant  with  GPA=  0^ 

and 


P 

1-P 

“d  and  *D  ln  (J2) 

A  A 

If  and  Bp  are 


advantaged  group  applicant  with  GPA«4 
advantaged  group  applicant  with  GPA-0. 
can  be  estimated  by  fitting  the  linear  regression  line. 

two  fitted  values,  then  the  utility  for  the  two  groups 


are 


Bd<9) 


“D  + 


6d  Ve) 


and 


ua(8)  -  uA(e). 
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Summary 

The  use  of  some  simple  utility  or  loss  functions  in  educational 
or  employment  evaluation  has  recently  been  studied  by  Petersen  (1976) 
Huynh  (1976,  1977),  Van  der  Linden  and  Mellenbergh  (1977).  Novick 
and  Lindley  (1978)  demonstrated  that  more  realistic  utility  functions 
can  be  easily  used  without  increasing  the  complexity  and  may  be 
preferable  in  some  applications. 

This  paper  illustrates  the  use  of  a  truncated  normal  or  a 
generalized  beta  cumulative  distribution  function  as  a  utility  function. 
They  are  more  flexible  chan  threshold  or  linear  functions.  If  a 
person's  utility  function  can  be  fitted  to  a  distribution  function 
the  analysis  of  utility  is  simple  and  easy.  For  example,  its 
derivative  is  the  familiar  density  function.  It  provides  the  rate 
of  the  increase  of  utility.  However,  one  should  not  force  a  person's 
utility  function  into  this  form  if  it  does  not  fit. 


References 


22 


Altham,  P.M.E.  Exact  Bayesian  analysis  of  a  two-by-two  contingency  table 
and' Fisher' s  "exact"  significance  test.  Journal  of  the  Royal  Statistical 
Society.  Series  B  31,  261-269,  1969. 


Phinno  n  T  Chen.  J.J.  and  Novick,  M.R.  Theory  and  Practice  for  the  use 

of’cut  scores  for  personnel  decisions.  Journal  of  Educational  Statistics, 
to  appear. 


Divgi,  D.  R.  Calculation  of  univariate  and  bivariate  normal  probability 
function.  The  Annals  of  Statistics  T_,  903-910,  1979. 

Gross,  A.  L.  &  Su,  W.  H.  Defining  a  'fair'  or  'unbiased'  selection 
model:  a  question  of  utilities.  Journal  of  Applied  Psychology 
60,  345-351,  1975. 

Huynh,  H.  Statistical  consideration  of  mastery  scores.  Psychometrika 
41,  65-78,  1976. 

Huynh,  H.  Two  simple  classes  of  mastery  scores  based  on  the  beta- 
binomial  model.  Psychometrika  42,  601-608,  1977. 

Huynh,  H.  A  class  of  passing  scores  based  on  the  bivariate  normal  model. 
Proceedings  of  the  American  Statistical  Association  (Social 
Statistics  Section).  310-313,  1979. 

Keeney,  D.  &  Raiffa,  H.  Decisions  with  Multiple  Objectives:  Preference 
and  Value  Tradeoffs,  New  York:  John  Wiley  and  Sons,  Inc.,  1976. 


Lindley,  D.  V.  A  class  of  utility  functions.  The  Annals  of  Statistics 
4j  1-10,  1976. 

Mosteller,  F.  &  Nogee,  P.  An  experimental  measurement  of  utility. 
Journal  of  Political  Economy  59.  371-404,  1951. 

National  Bureau  of  Standards  Tables  of  the  Bivariate  Normal  Distribution 
Function  and  Related  Functions.  Applied  Mathematics  Series  50, 
Washington,  D.  C. ,  Government  Printing  Office,  1959. 


Novick,  M.  R.  &  Jackson,  P.  H.  Statistical  Methods  for  Educational  and 
Psychological  Research,  New  York:  McGraw-Hill,  1974. 


Novick,  M.  R.  &  Lindley,  D.  V.  The  use  of  more  realistic  utility 

function  in  educational  applications.  Journal  of  Educational 
Measurement  181-192,  1978. 

Novick,  M. R  .  &  Lindley,  D.  V.  Fixed  state  assessment  of  utility 

functions.  Journal  of  the  American  Statistical  Association  74 
306-311,  1979.  “  " 

Petersen,  N.  S.  An  expected  utility  model  for  optimal  selection. 

Journal  of  Educational  Statistics  4^,  333-358,  1976. 

Pratt,  J.  W.  Risk  Aversion  in  the  small  and  in  the  large.  Econometrika 
32,  122-136,  1964. 


23 


Pratt,  J.  W. ,  Raiffa,  H. ,  &  Schlaifer,  R.  Introduction  to  Statistical 

Decision  Theory  (Preliminary  Edition),  New  York:  McGraw  Hill,  1965. 

Schlaifer,  R.  Analysis  of  Decisions  under  Uncertainty,  New  York: 

McGraw  Hill,  1969. 

Van  der  Linden,  W.  J.  &  Mellenbergh,  G.  J.  Optimal  cutting  scores 

using  a  linear  loss  function.  Applied  Psychological  Measurement 
1,  593-599,  1977. 


Weisberg,  H.  I.  Bayesian  comparison  of  two  ordered  multinomial 
populations,  Biometric  28,  859-867,  1972. 


Table  1  Sample  Data  for  the  Use  of 

Truncated  Normal  Utility  for 
Educational  Selection 


Group 

"i 

X 

S 

X 

Y 

S 

y 

r 

xy 

8 

Disadvantaged 

305 

13.47 

4.787 

1.68 

1.088 

.2772 

.063 

Advantaged 

2182 

19.03 

5.276 

2.07 

1.015 

.3732 

.072 

Source  of 

Data  - 

1971 

ACT 

Research 

Services 

used  in 

Petersen 

(1976). 
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Table  2 

The  Expected  Values  of  Two  Truncated 

Normal  Utilities 

Disadvantaged 

Advantaged 

V  =  1.5 

u  -  2 

0 

o 

a  *  2 

a  -  1.5 

0 

o 

2 

0.298 

0.197 

4 

0.324 

0.222 

6 

0.353 

0.250 

8 

0.382 

0.280 

10 

0.413 

0.312 

12 

0.444 

0.347 

14 

0.476 

0.383 

16 

0.509 

0.421 

18 

0.542 

0.460 

20 

0.575 

0.499 

22 

0.607 

0.538 

24 

0.639 

0.577 

26 

0.670 

0.615 

28 

0.700 

0.651 

30 

0.729 

0.686 

32 

0.756 

0.718 

34 

0.782 

0.749 

36 

0.806 

0.776 
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Table  4  The  Expected  Values  of  Two  Generalised  Beta  Utilities 


Score 


Disadvantaged 
a  »  2 

b  ■  4 


Advantaged 
a  -  2 

b  -  2 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 

13 

14 

15 


0.053 

0.140 

0.249 

0.366 

0.483 

0.594 

0.693 

0.779 

0.848 

0.902 

0.942 

0.969 

0.986 

0.995 

0.999 


0.020 

0.056 

0.108 

0.172 

0.245 

0.326 

0.412 

0.500 

0.588 

0.674 

0.755 

0.828 

0.892 

0.944 

0.980 
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Gambia 


Mo  null 


Bata 


9  point 


Table  6  Simulated  Studies  for  the 

Performance  of  normal  utility  flta  for 
Various  w  and  o  . 


2.00 

(0.038) 

2.51 

(0.053) 

2.97 

(0.042) 

0.50 

(0.015) 

0.50 

(0.017) 

0.49 

(0.009) 

1.97 

(0.123) 

2.48 

(0.131) 

2.85 

(0.172) 

0.99 

(0.110) 

1.01 

(0.117) 

0.95 

(0.090) 

2.02 

(0.305) 

2.44 

(0.340) 

2.77 

(0.262) 

1.59 

(0.555) 

1.46 

(0.304) 

1.35 

(0.254) 

1.88  (0.560) 

2.40 

(0.396) 

2.84 

(0.477) 

1.93 

(0.600) 

2.11 

(0.717) 

2.02 

(0.475) 

3.54  (0.240) 
0.88  (0.067) 

3.24  (0.362) 
1.18  (0.187) 

3.28  (0.500) 
1.62  (0.317) 


Least-squares  procedure  to  fit  a  normal  or  a  beta  cdf  utility 


If  we  write 


Chen  Che  least-wquare  estimate  of  b  is 

b<°>-  (X'<0)X<°W°>  <y-f<°>) 

t  can  be  estimated  by  the  interactive  process  from  the  above  equation, 
that  is 

,(n+1>  .  ,<n>+  b(n) 

-  ;<">+  <x'(",X(o))-1  X'(n)  (j  -  f(">). 


We  now  give  the  derivate  for  two  useful  parametric  utility  functions: 

2 

(1)  U(8)  is  a  normal  or  truncated  normal  cdf  with  mean  w  and  variance  o  , 

i.e.  0(8)  -  #(—*). 

Then 


8  ,  ,  m  ~  *1  ♦k~»1 

8  y  £ijk  (u’0)  a(fj-  \)  ”  °<V  V 


f...  (w,0)  -  —W— 1 - i - L 

*  0  1Jk  o2(®J  -  fj) 


(®k  -  u)»k  -  <e,-v>* 


’  -  V 


where  ^  -  ♦<-“-)  «nd  *(x) 


1  „  -V 


M 
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(2)  U(0)  is  a  beta  cdf  with  paraaeters  a  and  b,  i.e. 

■0 


U(0)  - 

P(a)r<b) 


x8"1  Cl  -  *)b“l 

a)r<b)  / 

-'A 


dx  .  Then 


f11k(.,b)  -  -In  -  2m 

3  a  ijh  utj  uJk 


_L_  f  (a  b)  .  !ii  -  -Us 
JbV‘)b)  uJk 


where 


9  9 

u..  -  f  X*”1  (1  -  x)W  dx,  V..  •  J  log  x  xa-1(l-x)b_1 

XJ  A  XJ  A 


dx 


and  w 


•  /"  ^  log  (1-x)  x*  1  (l-x)b  1  dx. 
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